Recent Posts

cpprb v10.1.0 can save & load transitions

From cpprb version 10.1.0, ReplayBuffer and its sub-classes can save and load transitions. from cpprb import ReplayBuffer rb1 = ReplayBuffer(256, {"obs": {"shape": 3}, "act": {}, "rew": {}, "done": {}}, next_of="obs")

GitLab CI/CD: Single job for multiple conditions

At a previous post, I used following configuration to achive manual or scheduled job at GitLab CI/CD. .docker_build_base: &docker_build image: docker:latest stage: build_image services: - docker:dind script: - docker login

ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject

When import cpprb, I got ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject. Recently, NumPy has changed its ABI at version

Hugo: Markdown rendering inside nested shortcodes

Hugo is a static site generator from markdown, which I use for this blog and project site. Hugo has a template mechanism, Shortcodes, which allows users to use complicated HTML.

Embed reveal.js slide into Hugo site

By utilizing custom shortcodes, we can embed pages including reveal.js slides into Hugo-powered site. First, create custom shortcodes at layouts/shortcodes/SlideInclusion.html <iframe src="{{.Get 0}}" width="1000" height="600" frameborder="0" allowfullscreen="allowfullscreen" allow="geolocation *; microphone




Bayes Neural Network for TensorFlow

Gym Notebook Wrapper

Wrapper for running and rendering OpenAI Gym on Jupyter Notebook


TensorFlow2 Reinforcement Learning


Replay buffer classes for reinforcement learning.


Survey of Experience Replay

Survey of Experience Replay

Hiroyuki Yamada


Hiroyuki Yamada got interested in IT (including programming, PC, etc.) during his graduate school. A few years later, he became an IT researcher at a Japanese company.

Encouraged by a friend, he started development of reinforcement learning software cpprb using Cython as a hobby at the beginning of 2019. This continuing development is still one of the biggest hobby in his life.

The author started this personal blog at the end of 2018. The motivation was to learn the way of making a site by utilizing GitLab CI/CD and a static site generator Hugo. Additionaly, he wanted to organize his own knowledge and experience based on the cpprb development.

H. Yamada’s GitHub Stats


  • Artificial Intelligence
  • Cloud Native Computing
  • C++
  • Python


Data Science

Statistics/Machine Learning


C++/Python (Cython)/shell script/JavaScript/Emacs Lisp/C#

Cloud Native Computing